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DETAILED ACTION 



Introduction 



1 . Claims 1-36 of the application have been examined. 



Drawings 



2. The drawings submitted on June 28, 2000 are accepted. 



Claim Rejections - 35 USC § 103 



3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 

4. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 
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4. 



Considering objective evidence present in the application indicating obviousness 
or nonobviousness. 



5. Claims 1, 3-10, 12, 13, 15-22, 24, 25, 27-34 and 36 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over Viniotis et al. (VI) ("Linear programming . . . Queueing systems", 
IEEE, 1988) in view of Schneider et al. (SC) ("Stochastic Production scheduling ... demand 
forecasts" IEEE, 1998). 

5.1 VI teaches Linear programming as a technique for optimization of queuing systems. 
Specifically, as per Claim 13, VI teaches solving stochastic control problems of linear systems in 
high dimensions (Page 652, CLl, Para 1; Page 653, CL2, Para 3); comprising: 

modeling a structured Markov Decision Process (MDP) (Page 652, CLl, Para 4; Page 
652, CL2, Para 6), wherein a state space for the MDP is a polyhedron in a Euclidean space (Page 
654, CL2, Lemma 2); 

one or more actions that are feasible in a state of the state space are linearly constrained 
with respect to the state (Page 653, CLl, Para 1 and Para 2; Page 652, CL2, Para 7); and 

building a value function for the state using representations that facilitate the computation 
of approximately optimal actions at any given state by linear programming (Page 653, CLl, Para 
9 to Page 654, CLl, Para 4; Page 652, CL2, Para 8). 

VI does not expressly teach a computerized apparatus for solving stochastic control 
problems of linear systems in high dimensions comprising a computer. SC teaches a 
computerized apparatus for solving stochastic control problems of linear systems in high 
dimensions comprising a computer (Page 2726, CLl, Para 3 and 4), as that allows the solution of 
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stochastic control problems of linear systems in high dimensions run faster and allows the user to 
generate the results with varying data (Page 2726, CLl, Para 3). It would have been obvious to 
one of ordinary skill in the art at the time of Applicant's invention to combine the method of VI 
with the apparatus of SC that included a computerized apparatus for solving stochastic control 
problems of linear systems in high dimensions comprising a computer type, as that would allow 
the solution of stochastic control problems of linear systems in high dimensions run faster and 
allow the user to generate the results with varying data. 

VI does not expressly teach logic performed by the computer, for modeling a structured 
Markov Decision Process (MDP). SC teaches logic performed by the computer, for modeling a 
structured Markov Decision Process (MDP) (Page 2726, CLl, Para 3 and 4), as that allows the 
solution of stochastic control problems of linear systems in high dimensions run faster and 
allows the user to generate the results with varying data (Page 2726, CLl, Para 3). It would have 
been obvious to one of ordinary skill in the art at the time of Applicant's invention to combine 
the method of VI with the apparatus of SC that included logic performed by the computer, for 
modehng a structured Markov Decision Process (MDP), as that would allow the solution of 
stochastic control problems of linear systems in high dimensions run faster and allow the user to 
generate the results with varying data. 

VI does not expressly teach logic performed by the computer, for building a value 
function for the state using representations that facilitate the computation of approximately 
optimal actions at any given state by linear programming. SC teaches logic performed by the 
computer, for building a value function for the state using representations that facilitate the 
computation of approximately optimal actions at any given state by linear programming (Page 
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2726, CLl, Para 3 and 4), as that allows the solution of stochastic control problems of linear 
systems in high dimensions run faster and allows the user to generate the results with varying 
data (Page 2726, CLl, Para 3). It would have been obvious to one of ordinary skill in the art at 
the time of Applicant's invention to combine the method of VI with the apparatus of SC that 
included logic performed by the computer, for building a value function for the state using 
representations that facilitate the computation of approximately optimal actions at any given state 
by linear programming, as that would allow the solution of stochastic control problems of linear 
systems in high dimensions run faster and allow the user to generate the results with varying 
data. 

VI does not expressly teach logic performed by the computer, for building one or more 
approximations from above and from below to a value function for the state using 
representations. SC teaches logic performed by the computer, for building one or more 
approximations from above and from below to a value function for the state using 
representations (Page 2722, CLl, Para 2; Page 2724, CL2, Para 6), as value function 
approximation is an effective technique for both deterministic and noisy scenarios (Page 2722, 
CLl, Para 2); and approximation allows solving large scale MDPs (Page 2722, CL2, Para 2). It 
would have been obvious to one of ordinary skill in the art at the time of Applicant's invention to 
combine the method of VI with the apparatus of SC that included logic performed by the 
computer, for building one or more approximations from above and from below to a value 
function for the state using representations, as value function approximation would be an 
effective technique for both deterministic and noisy scenarios and approximation allows solving 
large scale MDPs. 
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Per Claim 15: VI teaches that the action space and the state space are continuous and 
related to each other through a system of linear constraints (Page 652, CL2, Para 7; Page 653, 
CLl,Para 1), 

5.2 As per Claim 16, VI and SC teach the apparatus of Claim 13. VI teaches that the value 
function is convex (Page 652, CLl, Para 4). 

VI does not expressly teach that the logic further comprises efficiently learning the value 
function in advance and representing the value function in a way that allows for real-time choice 
of actions based thereon. SC teaches that the logic further comprises efficiently learning the 
value function in advance and representing the value function in a way that allows for real-time 
choice of actions based thereon (Page 2722, CLl, Para 2), as that allows producing a time- 
dependent action policy specifically built to match the current demand forecasts and other 
system states (Page 2724, CL2, Para 4). It would have been obvious to one of ordinary skill in 
the art at the time of Applicant's invention to combine the method of VI with the apparatus of 
SC that included the logic further comprising efficiently learning the value function in advance 
and representing the value function in a way that allows for real-time choice of actions based 
thereon, as that would allow producing a time-dependent action policy specifically built to match 
the current demand forecasts and other system states. 
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5.3 As per Claim 17, VI and SC teach the apparatus of Claim 13. VI teaches that the linear 
function is represented by piecewise linear and convex functions (Page 652, CLl, Para 4; Page 
654, CLl, Lemma 1). 

VI does not expressly teach the linear function is approximated both from above and 
from below by piecewise linear and convex functions. SC teaches the hnear function is 
approximated both from above and from below (Page 2722, CLl, Para 2; Page 2724, CL2, Para 
6), as value function approximation is an effective technique for both deterministic and noisy 
scenarios (Page 2722, CLl, Para 2); and approximation allows solving large scale MDPs (Page 
2722, CL2, Para 2). It would have been obvious to one of ordinary skill in the art at the time of 
Applicant's invention to combine the method of VI with the apparatus of SC that included the 
linear function being approximated both from above and from below, as value function 
approximation would be an effective technique for both deterministic and noisy scenarios and 
approximation allows solving large scale MDPs. 

Per Claim 18: VI teaches that the domains of linearity of the piecewise linear and convex 
functions are not stored explicitly, but rather are encoded through a linear programming 
formulation (Page 652, CLl, Para 4; Page 654, CLl, Lemma 1; Page 653, CLl, Para 9 to Page 
654, CLl, Para 4). 

5.4 As per Claim 19, VI and SC teach the apparatus of Claim 17. VI teaches that the 
domains of linearity of the piecewise linear and convex functions allow the functions to be 
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optimized and updated (Page 652, CLl, Para 4; Page 654, CLl, Lemma 1; Page 654, CLl, Para 
6 to Para 8). 

VI does not expressly teach that the domains of linearity of the piecewise linear and 
convex functions allow the functions to be optimized and updated in real-time. SC teaches that 
the domains of linearity of the functions allow the functions to be optimized and updated in 
real-time (Page 2722, CLl, Para 2; Page 2724, CL2, Para 6), as that allows producing a time- 
dependent action policy specifically built to match the current demand forecasts and other 
system states (Page 2724, CL2, Para 4). It would have been obvious to one of ordinary skill in 
the art at the time of Applicant's invention to combine the method of VI with the apparatus of 
SC that included the domains of linearity of the functions allowing the functions to be optimized 
and updated in real-time, as that would allow producing a time-dependent action policy 
specifically built to match the current demand forecasts and other system states. 

5.5 As per Claim 20, VI and SC teach the apparatus of Claim 13. VI does not expressly 
teach that the value function can be efficiently approximated both from above and from below. 
SC teaches that the value function can be efficiently approximated both from above and from 
below (Page 2722, CLl, Para 2; Page 2724, CL2, Para 6), as value function approximation is an 
effective technique for both deterministic and noisy scenarios (Page 2722, CLl, Para 2); and 
approximation allows solving large scale MDPs (Page 2722, CL2, Para 2). It would have been 
obvious to one of ordinary skill in the art at the time of Apphcant's invention to combine the 
method of VI with the apparatus of SC that included the value function being efficiently 
approximated both from above and from below, as value function approximation would be an 
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effective technique for both deterministic and noisy scenarios and approximation allows solving 
large scale MDPs. 

5.6 As per Claim 21, VI and SC teach the apparatus of Claim 13. VI does not expressly 
teach that the approximations can be repeatedly refined. SC teaches that the approximations can 
be repeatedly refined (Page 2725, CL2, Para 4), as that allows producing a time-dependent action 
policy specifically built to match the current demand forecasts and other system states (Page 
2724, CL2, Para 4). It would have been obvious to one of ordinary skill in the art at the time of 
Applicant's invention to combine the method of VI with the apparatus of SC that included the 
approximations being repeatedly refined, as that would allow producing a time-dependent action 
policy specifically built to match the current demand forecasts and other system states. 

5.7 As per Claim 22, VI and SC teach the apparatus of Claim 13. VI does not expressly 
teach that the value function can be efficiently approximated from above based on knowledge of 
upper bounds on the function at each member of a selected set of states. SC teaches that the 
value function can be efficiently approximated fi-om above based on knowledge of upper bounds 
on the function at each member of a selected set of states (Page 2722, CLl, Para 2; Page 2724, 
CL2, Para 6; Page 2725, CL2, Para 3), as value function approximation is an effective technique 
for both deterministic and noisy scenarios (Page 2722, CLl, Para 2); and approximation allows 
solving large scale MDPs (Page 2722, CL2, Para 2). It would have been obvious to one of 
ordinary skill in the art at the time of Applicant's invention to combine the method of VI with 
the apparatus of SC that included the value function being efficiently approximated fi-om above 
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based on knowledge of upper bounds on the function at each member of a selected set of states, 
as value function approximation would be an effective technique for both deterministic and noisy 
scenarios and approximation allows solving large scale MDPs. 

5.8 As per Claim 24, VI and SC teach the apparatus of Claim 13. VI does not expressly 
teach that the value function can be approximated successively. SC teaches that the value 
function can be approximated successively (Page 2725, CL2, Para 4), as that allows producing a 
time-dependent action policy specifically built to match the current demand forecasts and other 
system states (Page 2724, CL2, Para 4). It would have been obvious to one of ordinary skill in 
the art at the time of Applicant's invention to combine the method of VI with the apparatus of 
SC that included the value function being approximated successively, as that would allow 
producing a time-dependent action policy specifically buiU to match the current demand 
forecasts and other system states. 

5.9 As per Claims 1, 3-10, 12, 25, 27-34 and 36, these are rejected based on the same 
reasoning as Claims 13, 15-22, 24, supra. Claims 1, 3-10, 12, 25, 27-34 and 36 are computer 
based method and article of manufacture embodying computer logic claims reciting the same 
limitations as Claims 13, 15-22 and 24, as taught throughout by VI and SC. 

6. Claims 2, 14 and 26 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Viniotis et al. (VI) ("Linear progranaming ... Queueing systems", IEEE, 1988) in view of 
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Schneider et al. (SC) ("Stochastic Production scheduling ... demand forecasts", IEEE, 1998), 
and further in view of Dangat et al. (DA) (U.S. Patent 5,971,585). 

6.1 As per Claim 14, VI and SC teach the apparatus of Claim 13. VI does not expressly 
teach that the MDP comprises a supply chain plarming process. DA teaches that the MDP 
comprises a supply chain plaiming process (Abstract; CLl, L7-21; CL6, L5-9), as that provides a 
computerized decision support tool to generate a best can do (BCD) match between existing 
assets and demands across multiple manufacturing and other facilities based on process flows 
and business policies (Abstract), It would have been obvious to one of ordinary skill in the art at 
the time of Applicant's invention to combine the method of VI with the apparatus of DA that 
included the MDP comprising a supply chain planning process, as that would provide a 
computerized decision support tool to generate a best can do (BCD) match between existing 
assets and demands across multiple manufacturing and other faciUties based on process flows 
and business policies. 

6.2 As per Claims 2 and 26, these are rejected based on the same reasoning as Claim 14, 
supra. Claims 2 and 26 are computer based method and article of manufacture embodying 
computer logic claims reciting the same limitations as Claim 14, as taught throughout by VI, SC 
and DA. 

7. Claims 1 1, 23 and 35 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Viniotis et al. (VI) ("Linear programming . . . Queueing systems'*, IEEE, 1988) in view of 
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Schneider et al. (SC) ("Stochastic Production scheduling ... demand forecasts", IEEE, 1998), 
and further in view of Hedlund et al. (HE) ("Optimal control of hybrid systems", IEEE 1999). 

7.1 As per Claim 23, VI and SC teach the apparatus of Claim 13. VI does not expressly 
teach that the value function can be efficiently approximated from below based on linear 
functions that lie below the convex value function. HE teaches that the value function can be 
efficiently approximated from below based on linear functions that lie below the convex value 
fiinction (Page 3972, CLl, Para 1; Page 3973, CLl, Para 3; Page 3977, CLl, Para 1), as that 
would provide a lower bound on the optimal cost in terms of linear programming (Page 3972, 
CL2, Para 2). It would have been obvious to one of ordinary skill in the art at the time of 
Applicant's invention to combine the method of VI with the apparatus of DA that included the 
value function being efficiently approximated from below based on linear fiinctions that lied 
below the convex value function, as that would provide a lower boimd on the optimal cost in 
terms of linear programming. 

7.2 As per Claims 1 1 and 35, these are rejected based on the same reasoning as Claim 23, 
supra. Claims 1 1 and 35 are computer based method and article of manufacture embodying 
computer logic claims reciting the same limitations as Claim 23, as taught throughout by VI, SC 
and HE. 



Conclusion 
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8. The prior art made of record and not relied upon is considered pertinent to the 
Applicants' disclosure. 

The following patents and papers are cited to further show the state of the art at 
the time of Applicant's invention with respect to Markov Decision Process and its 
application to stochastic control and Supply chain planning: 

1. Lassere, "Average optimal stationary policies and linear programming in 
countable space Markov decision processes", IEEE 1992. 

2. Huang et al., "Decision support system for the management of an agile supply 
chain", U.S. Patent 6,151,582, November 2000. 

3. Heng et al., "Decision mle correction system and method", U.S. patent 
6,189,132, February 2001. 

4. Braun, "Method and system for the maximization of the range of coverage 
profiles in inventory management", U.S. Patent 6,341,266, January 2002. 

5. Dangat et al., "Method to provide common support for multiple types of 
solvers for matching assets with demand in microelectronics manufacture", 
U.S. patent 6,041,267, March 2000. 

6. Baseman et al., "Method for integrated supply chain and financial 
management", U.S. Patent 6,671,673, December 2003. 

7. Franssen et al., "Control flow and memory management optimization", U.S. 
patent 6,064,819, May 2000. 
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9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dr. Kandasamy Thangavelu whose telephone number is 
703-305-0043. The examiner can normally be reached on Monday through Friday from 
8:00 AM to 5:30 PM. 

If attempts to reach examiner by telephone are unsuccessful, the examiner's 
supervisor, Kevin Teska, can be reached on (703) 305-9704. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is 703-305- 
9600. 



K. Thangavelu 
Art Unit 2123 
March 6. 2004 




